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Abstract 

The concept and properties of eigenvalues and eigenvectors are presented in a concise and introductory manner. The 
importance of eigenvalues and eigenvectors in several areas is also briefly illustrated with respect to characterization of 
scalar field extrema, dynamical systems, Markov chains, and multivariate statistics. 


“Know thyself.” 

Delphic proberb. 


1 Introduction 

Eigenvalues, eigenvectors. These forever companion 
terms, appear recurrently in many areas of science and 
technology. Among their many applications, we have the 
inference of the type of extrema (e.g. maximum, mini¬ 
mum, saddle-point, etc.) of a multivariate function, char¬ 
acterization of the stability and convergence in dynamical 
systems, study of physical properties such as axes of iner¬ 
tia, development of important statistical methods (such as 
principal component analysis), detection of communities 
in network research, study of Markov processes... The 
list, which seems endless, continues to grow, helped by 
the fact that eigenvalues and eigenvectors can relate not 
only to matrices, but also to functions, dynamical states, 
graphs, etc. 

Though eigenvalues and eigenvectors (e.g. HJHIE]) are 
frequently used in several scientific and technological ar¬ 
eas, they are not always well-understood conceptually. 
Yet, they are ultimately intuitive and relatively simple, 
provided they are approached from the perspective of 
their properties, such as in preserving the inclination of 
vectors in linear transformation (see Figure [l]). 

The present work aims at providing a concise introduc¬ 
tion to the concept of eigenvalues and eigenvectors, also 
covering some of their most important and useful proper¬ 
ties, and including a few applications examples. We aimed 



Figure 1: An eigenvector it of a linear transformation T will not 
be modified by it other than by a scaling by A, its respectively 
associated eigenvalue. 


at addressing applications from the eigenvector and eigen¬ 
value perspective, in the sense that the respective concept 
and some important properties are presented as a prepa¬ 
ration for discussing some of the many representative ap¬ 
plications of eigenvalues and eigenvectors. 

Before proceeding, it is interesting to consider the ori¬ 
gin of these two terms. Since ‘values’ and ‘vectors’ are 
well-known mathematical entities, we focus on the term 
‘eigen’. Probably it derives from German, where ‘eigen 
’ means peculiar, proper, own, characteristic,self, inher¬ 
ent. Actually, the use of this term goes back at least 
to H. von Helmholtz’s ‘eigentone’, signifying the tones 
and resonances produced by vibrating bodies or systems 
(e.g. 0 ). 
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2 Some Basic Concepts 

An N x N matrix A has the general form: 


<M,l 

&1,2 

aij 



&2,i 

a 2,2 

. a 2 j 


«2,M 

i 

2 • 

a i,j 



(lN,l 

aw ,2 • 

• Ui,M 


UN,M 


A square matrix A is invertible if and only if its deter¬ 
minant is different from zero. 

The trace of a diagonal matrix A corresponds to the 
sum of the elements of its main diagonal. 

A real, square matrix A is orthogonal if and only if 
AA T = A T A = /, where I is the identity matrix , a diag¬ 
onal matrix with all elements equal to 1. 

The conjugate transpose, or Hermitian transpose of a 
complex matrix A is (A*) T = ( A T )* = A H . A complex, 
square Hermitian matrix A is such that A H A = AA H = I. 

The spectral radius R\ of a square matrix A is the max¬ 
imum magnitude of its eigenvalues. 

A square, nonnegative (all entries > 0 ) matrix A is said 
to be primitive if for some positive integer k > 0 we have 
that all elements of A k are positive (> 0 ) values. 

A graph can be associated to a real matrix A by making 
aij = 1 whenever we have a directed connection from 
node j to node i, and a^j = 0 otherwise. Such a matrix 
is called the adjacency matrix of the graph. 

Let A be a positive square matrix, and make each of 
its non-zero entries equal to 1 , and 0 otherwise. This 
matrix can be understood as an adjacency matrix defining 
a respective graph. 

A matrix A is irreducible if the resulting directed graph 
is strongly connected, i.e. given any two nodes i and j, 
there is a directed path from i to j. This can also be 
interpreted as: given any element aij of A there is an 
integer k > 0 such that the corresponding element pij of 
P = A k is positive (> 0 ). 

So, we have that every primitive matrix (nonnegative) 
is irreducible, with all its elements becoming nonnegative 
for the same power of k , but not vice-versa, implying that 
being primitive is a more restrictive conditions than being 
irreducible. Also, every positive square matrix is neces¬ 
sarily irreducible. 

The linear transformation of a vector x into a vector y 
can be expressed in matrix form as: 

y = Ax (2) 

where x = (x\,X2, ■ ■. , xn in $l N ) is a column vector 

Two matrices A and B are said to be similar if a third 


matrix P can be found such that: 

A = P- X BP ( 3 ) 

A diagonalizable real matrix A is similar to a diagonal 
matrix D, i.e.: 

A = p - 1 DP, ( 4 ) 

in case the matrix P exists, P~ X DP corresponds to the 
diagonalization of A. 

Let A be a symmetric matrix. Matrix A determines a 
respective quadratic form : 

q A {x) = x T Ax ( 5 ) 

The sign of the quadratic form of a real symmetric ma¬ 
trix A can be taken into account while classifying it into 
several types, as described in the following: 


Positive definite: 

q A (x) > 0 for any vector x ^ 0. 

Negative definite: 

q A (x) < 0 for any vector f ^0. 

Positive 

semidefinite: 

q A (x) > 0 for any vector x ^ 0. 

Negative 

semidefinite: 

q A {x) < 0 for any vector f/0. 

Indefinite: 

det(A) 7^ 0 and A is neither pos¬ 
itive definite or negative definite. 

Nul l- determinant 

case: 

If det(A) = 0, A can be indefinite 
or positive semidefinite or nega¬ 
tive semidefinite. 


Observe that a matrix being positive is not the same 
as being positive definite (and similarly for negative and 
negative definite). 

Let’s consider as an example that A = ^ ^ . Its 

quadratic form is: 


2 i 2 

x Ay . 


Observe that the quadratic form of A corresponds to a 
second order polynomial on the variables x and y. 

This polynomial can only take positive values, from 
which we conclude that A is positive definite , but is not 
positive as it includes the element zero (however, it can 
be said to be nonnegative). 

3 Eigenvalues and Eigenvectors of 
a Matrix 

We now proceed to defining and presenting some of the 
many important properties of the eigenvalues and respec¬ 
tive eigenvectors of a matrix A. 
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Given a matrix A , which we will considered as being 
associated to its respectively implemented linear trans¬ 
formation, we can define one of its eigenvalues A and re¬ 
spective eigenvector v as obeying: 

Xv = Av ( 7 ) 

Thus, an eigenvector v of the A is such that, when 
transformed by A , yields a new vector Xv that preserves 
the inclination of the original vector, though being scaled 
by the respective eigenvalue A. 

The eigenvalues of a matrix A can be determined by 
making the respective characteristic polynomial 

p(X) = det(A - XI) (8) 

equal to zero, and solving the resulting secular (or char¬ 
acteristic) equation. Then, one possible way to obtain the 
respective eigenvectors is by substituting the eigenvalues, 
one-by-one, into Equation [lO] and solving the respectively 
obtained systems of linear equations. 

It is also interesting to consider the equation 

Xf = f T A ( 9 ) 


defining the right-eigenvectors of A with respect to A 
(the previous ones are called left-eigenvectors). 

It can be shown that the eigenvalues A^ are the same for 
both Equations [lO] and [ 9 j but the respective eigenvectors 
v and r are generally not identical, unless A is symmet¬ 
ric. This can be verified by transposing Equation [ 9 ] and 


comparing it with Equation 10 


Let’s consider the determination of the eigevalues and 
respective eigenvectors of A = \ \ . We have: 


0 


-1 


p( A) = det(A — XI) — det 
1 - A 2 = 0 : 

For Ai = 1, we go back to Equation [To| 


1 - A 1 - A 

0 1 +A 

Ai = 1 and A2 = 


-1 


An eigenvalue A^ can also have a respective geometric 
multiplicity , 7 a (Ai), which is given as: 


7a(A i) = TV - rank(A - XJ) ( 11 ) 

It can be shown that: 

1 <7A(Ai) </M(Ai) < 7 V ( 12 ) 


If rank (A — Ai/) > 1 , implying that there are two or 
more linear independent eigenvectors with the same eigen¬ 
value, the respective Ai is said to be degenerate. In case 
all eigenvalues have multiplicity 1, the spectrum is said to 
be simple. 

The Perron-Frobenius theorem (e.g. [ 5 ]) provides im¬ 
portant information regarding the eigenvalue and eigen¬ 
vector properties of a square, nonnegative irreducible ma¬ 
trix A. Among other things, it implies that the respective 
R\ exists and is a simple root of the respective character¬ 
istic polynomial. We also have that there is an eigenvector 
associated to R\ that has strictly positive elements. 

A list of several additional eigenvalue and eigenvector 
properties are given in Table [l] while Table [ 2 ] includes 
some eigenvalue and eigenvector properties related to the 
positive or negative definiteness of a symmetric real ma¬ 
trix A. 

Given an TV x TV matrix A with respective non¬ 
degenerated eigenvalues A i = 1,2, ..., 7 V, with asso¬ 
ciated linearly-independent eigenvectors 77, we can define 
the following diagonal matrix: 


A = 


" Ax 
0 
0 
0 


0 

A2 

0 

0 


0 " 

0 

0 

Aat _ 


( 13 ) 


as well as the matrix of corresponding eigenvector: 


V = 


t 

Vl 


i 


t ... f ' 

v 2 ... Vn , 

l ... i _ 


( 14 ) 


Xv = v = Av 


V x - I5 Vy - 0 . 


( 10 ) 


The previous Equation [lO] can now be expressed in ‘ par¬ 
allel ’ manner (e.g. as: 


The reader is encouraged to find the eigenvector corre¬ 
sponding to A2 = —1. 

Given a matrix A, its eigenstructure can be understood 
as defined by its respective set of eigenvalues and eigen¬ 
vectors, while the set of eigenvalues is called the spectrum 
of A (typically, the diagonal of A is not considered when 
calculating its spectrum). 

Given an TV x TV matrix A , one of its eigenvalues A i may 
corresponds to a multiple root of the characteristic equa¬ 
tion, implying that eigenvalue to have respective algebraic 
multiplicity fiA{Xi). 


AV = VA. ( 15 ) 

Therefore, this equation expresses the interrelation¬ 
ships between all eigenvalues and eigenvectors of the orig¬ 
inal matrix A. 

If we left-multiply both sides of Equation 
we get: 

V~ 1 AV = A, ( 16 ) 

which is know as the eigendecomposition of A. Ob¬ 
server that this decomposition corresponds to the diago- 
nalization of A. 


10 by V 


-1 
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The above result allows us to obtain all the TV eigenval¬ 
ues (in case they exist) given matrix A and its respective 
eigenvectors. 

Observe that a matrix A may not have its inverse. In 
this case, its single-value decomposition can be used for 
respective diagonalization (e.g. pQ), being applicable even 
to non-square matrices. 

Now, if we right-multiply both sides of Equation [To| by 
U _1 , and assuming a set of eigenvalues given as A, we 
have: 

A = VAV-\ (17) 

which provides a method for designing a matrix A with 
pre-specified eigenvalues and eigenvectors. 

Given a row vector v with at least 2 components, it is 
easy to obtain a matrix A of which it is an eigenvector. 
This can be done as: 


A = vv T (18) 

If p is another vector orthogonal to v with the same 
dimension, a matrix A having v and p as eigenvectors, 
but a priori unspecified eigenvalues, can now be obtained 
as: 

A = vv T + pp T (19) 

Up to TV vectors TV X 1, each one orthogonal to all the 
others, can be combined in this manner so as to obtain 
an TV x TV matrix A having them as eigenvectors. 


4 Gershgorin Discs 

Proposed by the Russian applied mathematician 
S. A. Gershgorin [6], the discs, as well as the respective 
theorem that carry his name, provide an interest¬ 
ing resource for bounding the eigenvalues of complex 
matrices. 

Given an TV x TV complex matrix A, let the radius of 
the discs be calculated as: 


N 

Ri = \ a i,j\ (20) 

3 =b 

Observe that Ri is a real value corresponding to the 
absolute values of all elements along the i-th row of A. 
The Gershgorin disc for row i, G(i) is a closed circle de¬ 
fined in the complex plan with center at and radius 
Ri. Therefore, all such circles are centered at points along 
the real-axis. 

Gershgorin’s theorem states that all eigenvalues of A 
will necessarily be found inside the union of all the TV 
respective Gershgorin discs, which therefore bound the 
eigenvalues of a matrix. 


The following R algorithm can be used to visualize the 
Gershgorin discs of the TV x TV input matrix A. Observe 
that Nth is the angular resolution for plotting the discs. 


Algorithm 1 Gershgorin(A,N ) 

1. TVtA<-300 

2. dth A- 2 * pi/(Nth — 1) 

3. th <— seg(0, 2 * pT, dth) 

4. a <r- 7 

5. for(i in seg(l,TV)){ 

(a) R v- sum(abs(Ad[i, ])) 

(b) Gx <— R * cos(2 * pi * th) + A[i, i\ 

(c) Gy R * sin (2 * pi * th) 

(d) plot(Gx , Gy , xlim = c(—a, a),... 
ylim = c(— a, a), type = ”/”) 

(e) par (new = TRUE)} 


Figure [2] illustrates the Gershgorin discs (in salmon), 
as well as the actual three eigenvalues (green triangles, 
calculated by using an eigenvalue/eigenvector library), of 
the following matrix: 


A = 



-1 

2 

2 



( 21 ) 



Re 

Figure 2: The Gershgorin discs for the matrix A in Eq. |2l| shown 
in salmon, as well as the three respective actual eigenvalues, shown 
as green triangles. 


The above results allow some interesting interpretations 
and predictions about the eigenvalues of a matrix. For 
instance, we infer that the the higher the dispersion of 
the values of the elements along the diagonal of A , while 
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preserving all the other entries, higher also will be the 
dispersion of the discs along the real axis. 

In addition, the larger the absolute values along respec¬ 
tive rows of A, the higher the uncertainty, as inferred by 
this approach, in bounding the eigenvalues. However, we 
should not take higher bounding regions to necessarily 
imply larger eigenvalues. 

An interesting situation arises when we have a row i in 
which only the respective element corresponding to the 
diagonal, a^, is non-zero. In this case, one of the eigen¬ 
values will necessarily be equal to 

5 Extrema of Multivariate Func¬ 
tions 


We have, from property [P4] in Table[l| that this H(X) 
has eigenvalues Ai = 2 and A 2 = —2, implying the found 
critical point to be a saddle point 

6 Linear Dynamical Systems 

A homogeneous (no constant terms in the equations) lin¬ 
ear dynamical system involving N variables (JSt N ) with 
constant coefficients can be expressed as: 

x \(£) = ai t ixi(t) + ai, 2 x 2 (t) + • • • + ai,NXN(t) 
s m x 2 {t) = a 2 ,ixi(t) + a 2 , 2 x 2 (t) + • • • + a 12 ,NX N (t) 

, XN (t) = CiN,lXl(t) + a Ni2 X 2 (t) + . . . + aN,NXN{t ) 


In our first application example, we will briefly ad¬ 
dress the classification of the extrema of a scalar field 
VK x i, x 2 , • • •, xn) defined on . 

First, we identify the points yielding null respective gra¬ 
dient, i.e. 11 'S7'ip(x\i x 2 , ... ,x_/v)|| = 0. These so-called 
critical points are candidates for being extrema of yb. 
However, additional testing is required considering the 
eigenvalues of the Hessian matrix of A, namely: 


^0(P) 


d 2 ijj d 2r ijj d 2 ijj 

d 2 x dxdy dxdz 

d 2 ip d 2 V> d 2 ip 

dydx d 2 y dydz 

d 2 '0 d 2r ip d 2r ip 

dzdx dzdy d 2 z 


( 22 ) 


The following criteria can then be used while trying to 
classify the types of extrema of a scalar field: 


1. Positive-definite Hessian: All eigenvalues of 
H(x,y,z) are positive (i.e. A is positive definite) =>* 
(x,y,z) is a local minimum point ; 


where a*, j, i, j = 1, 2,..., N are real values. 

The system S can be placed in the equivalent matrix 
form: 


Xi(t) 


&l,l 


X2 (t) 

= 

&2,1 

«2,2 

. X N (t) _ 


_ CiN, 1 

CLN,2 


or, more synthetically: 


61, N 

62, N 

On,N 



Xi(t) 


X2 (t) 


. x N (t) _ 


(24) 


x{t) — Ax{t). 


(25) 


where A is an N x N matrix, and both x and x are 
column vectors. 

It can be shown (e.g. [7]) that the general solution of 
a linear system of ordinary differential equations at con¬ 
stant coefficients can be expressed as: 


2. Negative-definite Hessian: All eigenvalues of 
H(x,y,z) are negative (i.e. A is negative definite) 
==> (x,y,z) is a local maximum point ; 

3. Indefinite Hessian: The eigenvalues of H(x,y,z) 
are a mixture of positive and negative values => 
(x,y,z) is a saddle point ; 


x(t) — c\e Xlt vi + c 2 e X2t v 2 + ... + c N e XNt v N (26) 

where ci, C 2 ,..., cn are constants, Ai, A 2 ,..., Xn are 
the eigenvalues of A with respective eigenvectors 
V\ , v 2 , • • •, vn , provided they can be found. 

In case an initial condition xq is provided, the constants 
can be determined as: 


4. Otherwise: One or more null eigenvalues 
tional analysis is needed. 


addi- 


Observe the importance of the eigenvalues of H(^i) in 
identifying the types of extrema of a scalar field. 

Let’s illustrate the identification of the extrema of the 
scalar field = x 2 — y 2 . Its gradient is V(V’) = 2 xi — 2yj, 
which has null magnitude only when x = y = 0, so that 
we have X = (0,0) as the sole critical point of y). The 
Hessian of ^ is given as: 


H(X) = 


Cl 

C2 


- t t • 

V\ v 2 ■ 

• • t ' 

• • V N 

-1 

Xo,l 

X0,2 

_ Cn _ 


. 4- 4- • 

.. 1 _ 


_ Xq^n _ 


C = 


(27) 

As an example, let’s consider the solution of the linear 
ODE system with: 




" 1 

0 

3 " 

'2 0 ' 

(23) 

2 

0 

1 

_ 0 -2 

_ 0 

1 

3 _ 


(28) 
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We obtain A x = 3.81912..., A x = 0.09043... + 1.14062. ..i, 
and A 3 = 0.09043... — 1.14062...i, with respective eigen¬ 
vectors: 


vi 


0.63557... 

0.48922... 

0.59725... 


v 2 = 


0.15670... + 0.50738.. A 
0.80703... 

-0.24042... - 0.09425i 


V3 = 


0.15670... -0.50738 ..A 
0.80703... 

-0.24042... + 0.09425.. A 


(29) 


Considering the initial condition: 


x 0 


0 

-1 

1 


(30) 


we get the constants: 


C = 


0.87520... 

-0.88482... + 0.27487... i 
-0.88482... -0.27487... i 


(31) 


Figure [3] shows the three obtained solutions 
(t) for a period of time starting at 


t = 0. 


xl(t) 



Figure 3: The solutions of the ODE in the considered example un¬ 
folding along time. 


7 Markov Chains 

Markov chains constitute and interesting statistical ap¬ 
proach (e.g. 0) that is largely applicable. 


Let the following ordinary differential equation: 


x(t) = f(x(t)). 

Its discretization in time yields: 

which implies: 

x(t + At) ~ f(x, t)At + x(t), 


( 32 ) 


(33) 


(34) 


which corresponds to a first-order finite difference ap¬ 
proximation of the original equation, which can be used to 
estimate the solution of that equation by using numerical 
approaches. 

If we make At = 1 , and replace the approximation by 
an equality we get the new system: 


x{t + 1) = f{x{t)) + x(t), (35) 


Observe that this is not longer a good approximation of 
the original equation, corresponding to a distinct dynamic 
system. 

In /() corresponds to a linear transformation, we ob¬ 
tain: 

x(t + 1) = Bx(t) + x(t) - Ax(t ), (36) 

where A = B + I and in which the next state x t+1 
depends only on a linear combination of the current states 
corresponding to the entries of x(t). 

An initial condition can be expressed as x(t = 0). 

Let A be an Ax A real matrix. It is said to be stochastic 
or Markov if and only if it only contains nonnegative real 
entries and the sums of each of its columns is equal to 1 . 
As a simple example, we have: 


0.32 0.49 
0.68 0.51 

T T 


Observe that each of the rows of this type of matrix A 
can be understood as a normalized set of probabilities, in 
the sense that they add to 1 . 

It is also interesting to consider a row-stochastic ma¬ 
trix, which has all rows adding up to one. In this case, we 
shall call the previous type of matrix as being column- 
stochastic. A matrix that has these two properties is 
called a doubly stochastic matrix. 

Stochastic matrices can be sn to have all eigenvalues 
with magnitudes smaller or equal to 1. This follows from 
Gershgorin theorem: we have that each of the rows of a 
row-stochastic matrix adds up to 1 , so we know by that 
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theorem that they are bound in magnitude to 1. How¬ 
ever, as we also know that square matrices have the same 
left- and right-eigenvalues, we conclude that the right- 
eigenvalues of a stochastic matrix are also bound by mag¬ 
nitude 1. 

In addition, we have that every column-stochastic ma¬ 
trix has at least one left-eigenvector identical to 1 (row 
vector with all entries identical to 1), because 1A = 1 
effectively implements the sum of each of the columns of 
A. This vector is associated to the eigenvalue 1, so ev¬ 
ery row- and column-stochastic matrix have at least one 
eigenvalue equal to 1. 

If A is irreducible, we have from the Perron-Frobenius 
that the eigenvector associated to A = 1 can be placed in 
a form with strictly positive elements. This eigenvector 
will be associated to the stationary state of the respective 
Markov chain, also implying that every respective state 
will have a non-null probability. 

It should be kept in mind that a stochastic matrix A 
can have: (i) more than one eigenvalue equal to 1; (ii) 
eigenvalues equal to zero; (iii) negative eigenvalues; (iv) 
complex eigenvalues. 

If A is a stochastic matrix, Equation [36] defines a re¬ 
spective Markov chain on the states in x(t). 

Let’s consider that A is irreducible and regular. This 
effectively means that the state Xi(t ) associated to any 
node i of the graph representing A will, along time, influ¬ 
ences any of the other nodes with a non-null contribution. 

As already observed, the left-eigenvector associated to 
the eigenvalue 1 of A corresponds to the equilibrium 
or stationary distribution of probabilities of the Markov 
chain states, i.e.: 


0.9 



Figure 4: The graph associated to the stochastic matrix in the 
considered example. It is also interesting to imagine a uniformly 
random walk performed by a hypothetical agent along this graph, 
taking the outgoing links according to the respective transition prob¬ 
abilities. 


with corresponding eigenvectors: 


" -0.122... " 


’ 0.435.,. - 0.334.. A ' 

-0.967... 


-0.745... 

-0.193... 

; v 2 = 

0.036... +0.241. ..i 

_ - 0 . 110 ... _ 


. -0.273... +0.092 ..A . 


' 0.435... + 0.334 ...i ~ 


' -0.734... ' 

-0.745... 

; v 4 = 

0.600... 

0.036... -0.241 ...i 

-0.146... 

_ 0.273... -0.092 ..A _ 


0.280... 


Observe the coexistence of real and complex eigenvalues 
and eigenvectors. 

As expected, A has one eigenvalue identical to 1, with 
an associated real eigenvector corresponding to the sta¬ 
tionary state. This can be transformed into probabilities 
by normalizing pi = vi/sum(vi), which yields: 


Ap = p (37) 

Interestingly, this eigenvector does not depend on the 
initial state x(t = 0), and therefore has no ‘memory’ of 
the past dynamics or initial condition. 

Let’s consider the simple example of Markov chain pre¬ 
sented in Figure [4] 

The respective transition matrix A can be obtained as: 


0.3 

0 

0.1 

0.6 “ 

0.7 

0.9 

0 

0.1 

0 

0.1 

0.5 

0 

0 

0 

0.4 

0.3 _ 


' 0.0878... " 

_ _ 0.6940... 

Pl ~ 0.1388... 

. 0.0793... . 

In case we understand the transition probabilities in 
A as corresponding to a uniformly random walk on the 
respective system, the obtained distribution pi indicates 
that node 2 will be much more frequently visited than the 
others, followed by the third, first and forth nodes. 

Figure [5] illustrates the unfolding of the states values 
associated to the nodes along the discrete time steps t = 
0 , 1 , 2 ,..., 10 . 

8 Multivariate Statistics 


This matrix can be verified to be irreducible. 

The respective eigenvalues are Ai = 1, A 2 = 0.45461... + 
0.30132.. i, A 3 = 0.4546... - 0.3013.. .i, and A 4 = 0.0907..., 


The multivariate normal distribution (e.g. 0 ) is particu¬ 
larly important for modeling and trying to make predic¬ 
tions on a whole set of random variables (measurements). 
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t 

Figure 5: The values of the states probabilities (frequency of visits 
by a hypothetical agent) along the discrete time t from the initial 
condition xq = [0,0, 0,1]. Observe how, as a consequence of the 
interconnections and respective probability transitions, the value of 
the state of node 4, initially equal to 1, decreases quickly, as node 
2 progressively concentrates the density of visiting agents. 


Considering an N x TV domain, the multivariate normal 
probability density function with average vector ft and 
covariance matrix K can be expressed as: 

9H,k{X) = 

= (* -4 K ~ l (*-*)} (39) 

Observe the quadratic form of K~ x in the argument of 
the exponential above. 

Given N random variables Xi, i = 1,2, ...,7V, rep¬ 
resented as the random vector X, and their respective 
joint probability density function, the corresponding co- 
variances can be defined as: 

cov(X i ,X J )= f oo(X|-/i X J(X i -/i X .)p(X)dX (40) 

4—00 

The respective unbiased estimator, from M samples of 
the random variables is given as: 

1 M 

cov(X h Xj) « )T(W - p Xl ){Xj - HXj) (41) 

1 k =1 


The covariance matrix K can now be defined so that 
each of its elements kij = cov(Xi,Xj). This matrix has 
important properties, some of which are presented as fol¬ 
lows. 

First, we have that, as a consequence of its own defini¬ 
tion, that it is necessarily real and symmetric. Then, as 
the diagonal elements correspond to the respective vari¬ 
ance (i.e. var(Xi) = cov(Xi,Xi ), an average of squared 
values), all its elements are necessarily non-zero, and so 
is its trace, implying that the respective eigenvalues add 
to a nonnegative value. 


Indeed, the covariance can be shown to be positive 
semidefinite, i.e. its eigenvalues are all larger or equal to 
0, therefore its determinant is also nonnegative. 

In addition, by being symmetric, its eigenvalues are 
all real, and the eigenvectors V{ corresponding to distinct 
eigenvalues are orthogonal. 

Therefore, if we define the matrix: 


<— v{ 
<- v 2 


_ vN -> _ 


(42) 


it will orthogonal. We can apply this matrix on the 
original random vector, yielding a new random variable 
vector Y whose elements are linear combinations of the 
original random vectors, i.e.: 


Y = PX 


(43) 


which corresponds to a linear statistical transforma¬ 
tion known as discrete Karhunen-Loeve transform, which 
provides the basis for the Principal Component Analysis 
(PCA) methodology [10], ?]. PCA implements a rotation 
of the original coordinates axes so as to align the first axes 
with the directions of largest variation, as quantified by 
the respective variances. The obtained random variables 
result completely uncorrelated, and their variance is equal 
to eigenvalues associated to the respective axes. 


9 Concluding Remarks 

The present work presented, briefly and in an introduc¬ 
tory manner, the concept, properties and applications of 
eigenvalues and eigenvectors from ‘eigen-centered’ posi¬ 
tion. By starting with a review of some of their important 
properties, it was possible to discuss the subsequent ap¬ 
plications in a more integrated and systematic way. The 
interesting Gershgorin approach was also briefly outlined 
and illustrated. 

The addressed eigenvalues and eigenvectors applica¬ 
tions included scalar field extrema characterization, so¬ 
lution of linear dynamic systems at constant coefficients, 
Markov chains, as well as some aspects of multivariate 
statistics. 

The already large potential of theoretical and practical 
applications of eigenvalues and eigenvectors is being con¬ 
stantly further enhanced thanks to continuing advances 
in computer science, allowing respective calculations on 
matrices of ever increasing sizes. This opens up new 
prospects in theoretical and applied research. It is hoped 
that the covered presentation may motivate the reader to 
probe further in this interesting area. 
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Costa’s Didactic Texts - CDTs 


CDTs intend to be a halfway point between a 
formal scientific article and a dissemination text 
in the sense that they: (i) explain and illustrate 
concepts in a more informal, graphical and acces¬ 
sible way than the typical scientific article; and 

(ii) provide more in-depth mathematical develop¬ 
ments than a more traditional dissemination work. 

It is hoped that CDTs can also incorporate new 
insights and analogies concerning the reported 
concepts and methods. We hope these character¬ 
istics will contribute to making CDTs interesting 
both to beginners as well as to more senior 
researchers. 

Each CDT focuses on a limited set of interrelated 
concepts. Though attempting to be relatively 
self-contained, CDTs also aim at being relatively 
short. Links to related material are provided in 
order to complement the covered subjects. 

Observe that CDTs, which come with absolutely 
no warranty, are non distributable and for non¬ 
commercial use only. 

The complete set of CDTs can be found 
at: https://www.researchgate.net/project/ 

Costas-Didactic-Texts-CDTs, 
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Table 1: Some of the some properties of eigenvalues and eigenvectors 
of a matrix A. 


[PI]: Let a E $i, a / 0. The matrix 
a A has the same eigenvectors as A, 
while the eigenvalues are multiplied by a. 

[P2]: Let a E ?ft, a ^ 0. If v is an eigenvector 
of A, so is av. Therefore, the magnitude of an 
eigenvector cannot be uniquely specified. 

[P3]: The characteristic polynomial of an 

N x N matrix has degree TV, and so 
has N roots, which may appear with multiplicity. 

[P4]: The eigenvalues of a diagonal matrix 
are the elements of its diagonal. 

[P5]: The trace of a matrix is equal to the sum 
of its eigenvalues. 

[P6]: The product of the eigenvalues of a matrix 
is equal to its determinant. 

[P7]: If A is an eigenvalue of A, 
it is also an eigenvalue of A T . 

[P8]: The eigenvalues of A r , for a 
positive integer r, are (A^) r . 

[P9]: If A is an eigenvalue of A , 
then A -1 is an eigenvalue of A ~ l . 

[P10]: Eigenvectors associated to distinct 
eigenvalues are linearly independent. 

[Pll]: The eigenvalues of a real matrix 
are real or come in conjugate pairs. 

[PI2]: All the eigenvalues of a 
complex /real matrix A which is 

Hermitian /symmetric matrix are real. 

[PI 4]: If a comp lex /real matrix A is 
Hermitian /symmetric, all its eigenvectors 
are linearly independent and mutually orthogonal. 

[PI5]: If a complex /real matrix A is 
unitary /orthogonal, then all its 
eigenvalues have absolute value equal to 1. 

[PI6]: A is invertibe if and only if 
all its eigenvalues are non-null. 


Table 2: Some positive and negative definiteness-related properties 
of a symmetric real matrix A. 


Positive definite: 

If and only if all its eigenvalues 
are positive. 

Negative definite: 

If and only if all its eigenvalues 
are negative 

Positive 

semidefinite: 

If and only if all its eigenvalues 
are nonnegative (> 0). 

Negative 

semidefinite: 

If and only if all its eigenvalues 
are non-positive (< 0). 

Indefinite: 

If and only it has both positive 
and negative eigenvalues. 
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